Not Applicable
Not Applicable
Not Applicable
1. Technical Field
The field of the present invention is input devices for computing devices and a method of using the input device. More particularly, the present invention relates to the control of a computer device with voice inputs.
2. Background Art
Many modern computing devices have a graphical user interface where a user interacts with the computer using a graphical input device. For example, the well-known and ubiquitous personal computer is most often configured to operate the popular xe2x80x9cMicrosoft Windows(trademark)xe2x80x9d graphical user interface. The graphical user interface and graphical applications operating thereupon typically require the user to select graphical display elements to control and command the personal computer. Therefore, the personal computer is most conveniently operated using a graphical input device.
In this manner, a user typically uses the graphical input device for directly controlling the graphical elements of the graphical user interface and to activate commands. In some instances, the graphical input device works in conjunction with a keyboard to more efficiently enter commands and control a graphical application. There are many different types of graphical input devices in use today. Some of the most popular are graphics tablets, touch screens, digitizers, light pens, and trackballs. However, by far the most popular graphical input device is the standard computer xe2x80x9cmousexe2x80x9d.
The standard mouse comprises a housing to be manipulated by the palm of the hand. The bottom side of the mouse has a roller ball for contacting a flat horizontal surface. As the user propels the mouse along the flat horizontal surface, the roller ball provides relative left and right movements to mouse circuitry in the computing device. These relative left and right movements are communicated to the graphical driving system of the personal computer, where the relative mouse movements are converted to relate to the graphical coordinate system used on the Windows(trademark) graphical user interface. In such a manner, left and right movements of the mouse across the flat horizontal surface are converted into left and right movements across the display for the computing device.
Most often, the movement of the mouse is represented on the display of the computing device by the position of a graphical cursor. The graphical cursor may take many shapes or forms such as an arrow or a cross hair. Thus, by moving the mouse, the cursor may be made to move to any location on the display and point to any location.
The typical mouse further has buttons for providing additional control and command functions. A mouse may be configured with any number of buttons, however most applications today are configured to operate best with a two button mouse. Typically, the mouse buttons are referred to as the xe2x80x9cleft mouse buttonxe2x80x9d and the xe2x80x9cright mouse buttonxe2x80x9d relative to the front of the mouse.
Each mouse button has particular functionality as defined by the operating system. The Windows(trademark) operating system usually provides that the mouse functionality be done with the left mouse button. For example, the left mouse button may be used to input commands to the Windows(trademark) operating system. In such a manner, the mouse is used to move a graphical cursor to a command name on a Windows(trademark) command line. The left mouse button is depressed once (a click). After the mouse button has been clicked, a drop down menu appears and a sub-menu of available commands appears. The user then actuates the mouse to move the pointer (cursor) to the desired command and then clicks the left mouse button. Once the command has been clicked with the left mouse button, the command is performed.
Additionally, several commands are activated by depressing the left mouse button twice in rapid succession (called a double click). In many programs, such as a Windows(trademark) word processor, double clicking on an individual word causes that word to be selected for additional commands. For example, a user may double click on a word and then select the xe2x80x9cdeletexe2x80x9d key. Thereby the whole selected word is deleted in a single keystroke.
The right mouse button also has been assigned typical functionality, especially in more recently developed applications. For example, in many Windows(trademark) word processing systems, the right mouse button automatically accesses a menu structure having the commands the user is most likely to next use. As described above, a user may select a word by double clicking on it with the left mouse button. Then, while still pointing at the selected word, the user may single click on the word with the right mouse button. Thereby a menu appears giving format options, editing options, and display options. The use of the right mouse button eases the interaction with the graphical user interface.
The mouse is also used to provide more sophisticated control of the graphical user interface. For example, the mouse may be used to select large areas of text or other graphical items. In such a manner, for example, the user first positions the graphical cursor in an upper left-hand corner of the block desired to be selected, clicks and holds the left mouse button and then drags the mouse to cause the image to move a lower right-hand position. An imaginary rectangular box is created extending from the upper left-hand position to the lower right-hand position. Thereby, when the left mouse button is released, the text or other items within the imaginary box are selected for further operation.
The mouse provides additional features in particularly graphical intensive applications. For example, in a drafting package, the mouse may be used to draw or manipulate lines or other graphical structures. Further, although the Windows(trademark) operating system generally defines the typical uses of the mouse, each application program may extend the functionality of the mouse.
With the proliferation of graphical user interfaces on computing devices, these graphical user interfaces are showing up in locations where it may be inconvenient or otherwise undesirable to use a mouse. For example, computers including personal computers are currently used in dusty or dirty environments, such as by employees in a vehicle service facility, or in a facility where manufacturing process control systems are performed.
Such personal computers often reside within the service or manufacturing facility, and are operated by service or manufacturing personnel, or laborers. Often, such environments are hostile to the use of a mouse. In particular, a mouse requires a clean working environment since a dirty working environment causes grime to accumulate on the roller ball, thereby causing the mouse to fail. In such a manner, a mouse used in a service or manufacturing facility is prone to failure, and therefore the use of a mouse is impractical.
Further, it is often difficult and inconvenient for the service and manufacturing personnel to discontinue performing their assigned tasks, and provide the required input to the computer system. Therefore, information that should be entered into computer may not be entered, or the servicing or manufacturing personnel may be interrupted from their normal assignment to provide input to the personal computer. Either way, the operation of the facility is made less efficient. In such a manner the use of a mouse is inefficient, at best.
Indeed, there are numerous environments where it is inconvenient or impractical to use a mouse. In these circumstances the user must provide undesirably cumbersome and awkward alternatives, or forego entirely the use of some system or application.
Therefore, it would be highly desirable to have a new and improved graphical input device for use with a graphical user interface for enabling more conveniently and efficiently a user to input graphical information and commands. However, as already indicated, any substitute graphical input device must fully implement the functionality of a conventional mouse to interact efficiently and completely with modern graphical operating systems and application programs. If the graphical input device does not emulate all mouse functions, the user may become xe2x80x9cstrandedxe2x80x9d in an application and be forced to resort to using the mouse or otherwise work in an inconvenient manner.
Thus, it would be highly advantageous to have a new and improved graphical input device, which can emulate a prerecorded sequence of computer mouse functions, and which can be utilized in almost any environment including dusty and dirty service and manufacturing environments. Alternately, any specific individual mouse-like input can be directly entered and performed, such as controlling the movement of a cursor, a point and click, a double click, a drag and all other mouse functions, in this case without the need for pre-recording or storing any commands. In this regard, in order to be fully mouse compatible, such an improved input device should be capable of interacting directly and immediately with the computer to execute any and all desired pointing functions in a similar manner as a mouse. Such an improved device can not only perform the direct and immediate control of a cursor, but also be operated in hostile environments where the operation of a conventional mouse would not be recommended.
It is therefore an object of the present invention to provide a method and a system for controlling a computing application program in a reliable and efficient manner.
It is another separate object of the present invention to provide an efficient technique for overlaying a voice command mouse emulation method, on top of any mouse driven application program without the need to modify the application program.
It is another separate object of the present invention to provide for the full emulation of mouse functionality, including relative cursor movements keyed by a sequence of audible cursor movement control commands.
It is another separate object of the present invention to provide a very robust voice input system that can be used in any industrial or commercial environment so that the background noise, the location of the operator, the inability of using hands and/or eyes, and the need for unlimited mobility does not affect the system operation.
Briefly, in accordance with the present invention, there is provided a new and improved voice input system and method which provide the ability to reliably enter voice activated mouse emulation commands.
A voice input system has a microphone coupled to a computing device, with the computing device typically operating a graphical application. A user speaks voice commands into the microphone, with the computing device operating a voice command module that interprets the voice command and causes the graphical or non-graphical application to be commanded and controlled consistent with the use of a physical mouse. In one form of the invention, voice commands are overlayed on normal command points of the application program. Such voice commands are stored in a separate voice command repository, with an edit mode provided to create new voice command sequences and to edit existing voice command sequences.
The inventive system enables an application program to be controlled by injecting interactive voice command dialogs (Macros) which can reliably emulate mouse and/or keyboard events. These Macros include the capability of providing a separate visual and/or audio feedback or prompting response to a voice command so that the operator is certain that his/her voice command was not misinterpreted by the voice recognition software. Moreover, in the case of a prompt feedback, the operator may choose to allow or deny the impending mouse event. In order to provide a robust system, a background noise canceling microphone, with wireless transmission between the operator and the computer, is employed with a synthesized sound feedback for each command.